Skip to content

Delegate RunService.GetActionData to DataProxyService #7284

Merged
pingsutw merged 1 commit into
flyteorg:v2from
BarryWu0812:#7253
Apr 28, 2026
Merged

Delegate RunService.GetActionData to DataProxyService #7284
pingsutw merged 1 commit into
flyteorg:v2from
BarryWu0812:#7253

Conversation

@BarryWu0812
Copy link
Copy Markdown
Contributor

@BarryWu0812 BarryWu0812 commented Apr 26, 2026

Delegate RunService.GetActionData to DataProxyService to remove duplicate implementation

Closes #7253

Why are the changes needed?

Today both services:

  • Hold a storage.DataStore and implement parallel reads of inputs.pb / outputs via errgroup.

  • Resolve storage URIs from the action record (DataProxy goes through RunService.GetActionDataURIs; RunService reads them directly from the DB).

  • Must stay in lock-step whenever we evolve the storage layout, artifact types, or error semantics (not found, partial results, trace vs task actions, etc.).

This is duplicate surface area with no good reason now that DataProxy owns data access. It also makes the storage-layout contract implicit in two places.

What changes were proposed in this pull request?

Refactor RunService.GetActionData into a thin shim that forwards to DataProxyServiceClient.GetActionData:

  • Inject a dataproxyconnect.DataProxyServiceClient into RunService (likely the in-process one for the all-in-one binary).

  • In GetActionData:

    1. Validate the request as today.
    2. Call DataProxyService.GetActionData with the same action_id.
    3. Convert the dataproxy.GetActionDataResponse to workflow.GetActionDataResponse (schemas need to be checked for 1:1 compatibility — they currently both use task.Inputs / task.Outputs so this should be trivial).
  • Remove the direct dataStore / repo reads from this RPC once delegation is in place. The URI resolution logic already lives behind GetActionDataURIs, which DataProxy calls.

  • Keep the runs RPC registered so existing clients keep working.

How was this patch tested?

Labels

Please add one or more of the following labels to categorize your PR:

  • added: For new features.
  • changed: For changes in existing functionality.
  • deprecated: For soon-to-be-removed features.
  • removed: For features being removed.
  • fixed: For any bug fixed.
  • security: In case of vulnerabilities

This is important to improve the readability of release notes.

Setup process

Screenshots

Check all the applicable boxes

  • I updated the documentation accordingly.
  • All new and existing tests passed.
  • All commits are signed-off.

Related PRs

Docs link

…cate implementation

Signed-off-by: Barry Wu <a0987818905@gmail.com>
if err := s.dataStore.ReadProtobuf(groupCtx, inputRef, resp.Inputs); err != nil {
logger.Errorf(groupCtx, "GetActionData: failed to read inputs from %s: %v", inputRef, err)
return connect.NewError(connect.CodeInternal, fmt.Errorf("failed to read inputs from %s: %w", inputRef, err))
if !storage.IsNotFound(err) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we return an error if the input is not found?

Copy link
Copy Markdown
Contributor Author

@BarryWu0812 BarryWu0812 Apr 28, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/flyteorg/flyte/pull/7284/changes#diff-5698a432711a8b677ea5e6d843f3a66a0c8cff4ebfaa6d262b844456364b46fbL11-L790

I saw the original logic is to skip the error that input is not found, so I kept the same logic.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah, i see

}
resp.Outputs = &task.Outputs{
Literals: inputsOrOutputs.GetLiterals(),
if !storage.IsNotFound(err) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

@pingsutw pingsutw added this to the V2 GA milestone Apr 27, 2026
@pingsutw pingsutw merged commit 6012f42 into flyteorg:v2 Apr 28, 2026
20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants